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LONG-TERM  GOAL 

Our  long  term  goal  is  to  provide  the  Navy  with  a  robust  automated  target  cueing  and  identification 
capability  for  use  with  high  resolution  underwater  imaging  sensors  such  as  Raytheon’s  laser  line  scan 
system.  These  electro-optical  sensors  operate  at  standoff  ranges  to  survey  large  areas  of  the 
underwater  environment  in  search  of  naval  targets  of  interest  and  are  currently  being  incorporated  into 
sonar  towbodies  to  create  high  probability  of  detection  and  low  false  alarm  rate  mine  countermeasure 
(MCM)  systems. 

OBJECTIVES 

The  objectives  of  this  year’s  effort  are  two-fold.  First,  we  have  set  out  to  determine  and  demonstrate 
the  ability  of  a  multi-spectral  laser  line  scan  sensor  to  optically  excite  and  exploit  fluorescence 
phenomena  and  thereby  facilitate  the  identification  of  underwater  mines  and  mine-like  objects.  Of 
particular  interest  are  targets  that  have  been  in  a  shallow  water  littoral  environment  for  extended 
periods  of  time  and  have  thus  been  subject  to  degrading  processes  such  as  corrosion  and  bio-fouling. 

Our  second  objective  is  to  adapt  and  assess  the  ability  of  an  existing  automatic  target  cueing  and 
recognition  approach  to  separate  and  classify  underwater  targets  of  interest  from  the  natural 
background  in  either  single  (i.e.  reflectance)  or  multi-channel  (i.e.  flourescence)  laser  line  scan 
imagery.  This  objective  ties  this  year’s  effort  into  our  long  tenn  MCM  goals  independent  of  any 
immediate  conclusions  that  may  be  drawn  about  the  utility  of  exploiting  fluorescence  imagery. 

APPROACH 

Survey  Phase: 

The  first  phase  of  our  project  was  a  survey  of  a  target-rich  shallow  water  environment  approximately 
10  miles  southwest  of  Key  West,  FL.  The  survey  was  conducted  to  acquire  multi-channel  fluorescence 
imagery  of  submerged  mine-like  objects  of  interest,  presumed  to  be  defused  WWII  mines  laid  to 
protect  the  local  shipping  lanes.  The  survey  was  performed  on  the  Sea  Diver,  supplied  by  the  Harbor 
Branch  Oceanographic  Institute  (HBOI).  The  Fluorescence  Imaging  Laser  Line  Scanner  (FILLS)  used 
to  acquire  the  imagery  was  installed  in  a  Coastal  Systems  Station  (CSS)  developed  and  operated  active 
depressor  towbody  that  was  towed  behind  the  Sea  Diver.  The  towbody  also  housed  forward,  and 
down-looking  sonars  that  provided  for  wide  area  target-like  object  locating  and  navigational  control. 

In  addition,  in  situ  water  samples  and  measurements  to  characterize  the  water  column  were  made  by 
Dr  Dave  Phinney  and  his  group  from  Bigelow  labs. 
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The  survey  consisted  of  multiple  tracks  over  a  highly  bio-fouled  mine  and  adjacent  anchor  that  had 
earlier  been  located,  marked,  and  video  documented  by  a  dive  team.  The  team  also  emplaced  a 
calibration  panel,  prepared  by  Dr.  Charles  Mazel  of  PSI,  containing  four  two-foot  squares  of  green, 
yellow,  and  red  fluorescing  materials.  The  panel  was  situated  such  that  it  would  be  imaged  along  with 
the  mine  to  assess  the  water  column  impact  on  FILLS-induced  fluorescence.  Additional  survey  tracks 
were  made  over  an  area  known  to  contain  multiple  minefield  lines.  The  objective  of  these  passes  was 
to  obtain  multiple  target  samples  to  better  define  this  class  of  objects  in  support  of  algorithm 
development. 

Data  Analysis  Phase: 
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1.  Multi-Stage  Object  Identification  Processing  Flow  Diagram 

The  second  phase  of  our  project  commenced  in  late  August  and  is  scheduled  to  continue  through 
December  1999.  In  this  phase,  we  are  adapting  Raytheon  proprietary  target  cueing  and  object 
identification  algorithms  to  detect  and  classify  mine-like  objects  in  single  and  multi-channel  laser  line 
scan  imagery.  After  the  adaptation  is  completed,  we  will  process  imagery  collected  at  Key  West  and  at 
other  locations  to  gain  an  improved  understanding  of  the  issues  associated  with  detecting  and 
identifying  underwater  objects  over  a  range  of  attenuation  and  scattering  conditions. 

The  target  cueing  approach  for  this  project  is  an  adaptation  and  extension  of  the  image  processing  suite 
developed  to  detect  and  extract  features  from  mine-like  objects  for  the  Anny’s  Airborne  Standoff 
Minefield  Detection  (ASTAMIDS)  program.  The  approach  automatically  detects  objects  with  shapes 
that  can  be  approximated  by  an  ellipse  within  a  range  of  target-like  sizes  and  eccentricities.  The 


processing  stages  include:  mean  downsampling  to  lessen  downstream  throughput  and  improve  the 
signal-to-noise  ratio;  Sobel  filtering  with  adaptive  local  binarization  to  yield  object  outlines;  geometric 
feature  extraction  to  detennine  object  size,  shape,  and  fit  to  an  ellipse;  and  a  simple  decision  rule  that 
operates  on  the  feature  set  and  incorporates  rudimentary  knowledge,  such  as  the  size  and  shape  of 
known  mines.  Typically,  this  processing  is  done  in  real-time  and  the  results  are  presented  to  the  user  as 
an  icon  or  pointer  overlay  on  an  image  display.  The  overlays  draw  the  viewer’s  attention  to  mine-like 
objects  and  allow  more  time  for  a  target  or  no  target  decision  before  the  next  scene  is  presented. 

In  our  long  term  goal,  the  cuer  will  handoff  regions  of  interest  to  a  more  computationally  intensive 
object  identifier  to  fonn  a  fully  automated  target  ID  processing  chain.  The  object  identifier  will 
definitively  declare  a  target-like  object  (from  the  cuer)  to  be  either  a  target  or  non- target  and  will, 
ideally,  eliminate  the  need  for  a  subjective  human  decision.  Our  approach  to  complex  environment 
object  identification  is  a  vision  system,  whose  processing  framework  is  depicted  in  Figure  1,  that 
incorporates  multiple  classification  strategies.  Each  strategy  has  strengths  as  well  as  weaknesses  but 
they  are  designed  to  compliment  each  other  well  when  used  in  concert  within  the  framework. 

To  this  point,  a  hybrid  generic-  and  case-based  modeling  approach  incorporating  elements  of  the 
Exemplar,  Generic,  and  Abstraction  strategies  (see  below)  has  been  implemented  with  some  success 
for  a  limited  set  of  naturally-occurring  underwater  objects  such  as  sponges  and  corals.  In  this  project, 
we  will  further  develop  those  strategies  that  are  deemed  critical  to  support  mine  detection  based  on  an 
analysis  of  data  collected  from  Key  West  and  other  deployments.  A  brief  discussion  of  the  classifier 
strategies  are  given  in  the  following  paragraphs. 

Exemplar-Based  Strategy  (viewer-centered):  Classification  is  achieved  by  determining  the  object 
exemplar  most  similar  to  the  new  item.  The  system  begins  by  using  superficial  (raw)  features  to  make 
an  initial  hypothesis  and  verifies  the  hypothesis  by  finding  an  exemplar  from  that  category  that 
matches  the  new  findings  well 

Generic  Recognition  Strategy  (object-centered):  To  complement  the  weaknesses  of  the  exemplar- 
based  recognition  strategy,  object  recognition  must  rely  on  coarse,  qualitative  models  representing 
classes  of  objects.  The  generalization  process  with  ‘simple’  operations  and  inferential  reasoning  can 
achieve  the  ‘near’  viewpoint-invariant  3D  features  from  the  readily  perceivable  (superficial)  features. 

Learning-Based  Strategy:  In  the  human  visualization  system,  it  is  hypothesized  that  descriptions  of 
objects  built  from  features  are  learned  from  examples.  Experts  in  the  field  believe  that  machine 
learning  is  driven  by  the  notion  that  algorithms  and  systems  can  improve  their  performance  with  time. 
To  evaluate  the  effectiveness  of  the  generalized/hypothesized  features  for  discriminating  objects  and 
improving  the  vision  system’s  efficiency,  the  following  learning  elements  are  essential:  compiling 
compiling;  updating  feature  significance  levels;  identifying  redundant  features;  evaluating  the  affects 
of  near/far  misses  and  too  few/many  activated  objects;  updating  uncertainty  and  (object)  variety  for 
fuzzy  representation  of  features;  and  ranking  features  by  discrimination  power  and  by  their 
ease/difficulty  of  computation. 

Evidence-Based  Strategy:  In  this  strategy,  object  recognition  is  regarded  as  the  task  of  obtaining  a 
subset  of  image  parts  whose  properties  and  relations  satisfy  a  set  of  constraints  for  rapid  object 
invocation.  Rather  than  using  static,  pre-specified  models,  a  vision  system  would  be  able  to  derive 
specific  model  information  dynamically  from  sensor  input  by  using  a  “teach-by-example”  technique, 
such  as  a  learning-based  strategy.  In  this  framework,  recognition  is  based  on  the  strength  of  collective 


evidence.  The  hypothesis  is  that  discrimination  power  is  induced  by  a  collective  set  of  diverse  and 
abundant  (individually)  weak  features. 

Abstraction-Based  Strategy:  Ideally,  the  multiple  techniques  discussed  earlier  must  have  stages  that 
are  closely  integrated.  The  results  of  each  stage  would  provide  feedback  to  refine  the  techniques  used 
at  a  previous  stage.  In  the  abstraction-based  paradigm,  a  hypothesis-verification  strategy  is  used  to 
extract  abstractions  from  the  data  {features  based  on  explicit  assumptions  made  on  data).  To  detennine 
whether  the  assumptions  are  applicable,  the  abstractions  are  instantiated  and  then  tested  for 
verification.  The  successful  abstractions  are  then  incorporated  into  the  data  set  to  modify  the  model- 
base  with  improved  context  infonnation.  To  support  the  system’s  concept  formation  and  adaptation 
capability,  in  relation  to  the  uncertainties  of  each  of  the  3D  features’  accuracy  and  discrimination 
power,  an  elaborate  feedback  mechanism  is  devised.  This  mechanism  is  essential  because  of  its 
complementary  nature  to  the  generalization  process  embedded  in  the  generic  recognition  strategy. 

WORK  COMPLETED 

Survey  Phase: 

The  Key  West  Survey  was  executed  on  the  nights  of  June  13th  and  14th,  1999.  The  most  useful 
imagery  was  collected  on  the  second  night  as  water  conditions  were  slightly  improved  and  the  FILLS 
system  operated  non-stop  between  9pm  and  2:30am.  Data  was  recorded  continuously  by  CSS  while 
17  snapshots  were  recorded  off  Raytheon’s  waterfall  display.  The  snapshots  included  the  known  mine 
and  panels,  other  mine-like  objects,  and  interesting  examples  of  natural  and  manmade  clutter. 

Data  Analysis  Phase: 

This  phase  of  the  project  was  delayed  until  late  August,  1999  to  accommodate  the  completion  of  the 
Raytheon  funded  development  of  the  object  identification  algorithm.  Under  this  contract,  the 
algorithm  code  has  been  upgraded  to  incorporate  a  technique  for  complex  object  modeling  that 
describes  the  topology  of  a  complex  object  as  a  set  of  basic  shapes  whose  contact  with  each  other  is 
defined  in  tenns  of  location,  penetration,  and  orientation.  This  technique  was  added  to  improve  the 
classifier’s  functionality  while  trying  to  identify  mines  with  large  localized  bio-growths  that 
significantly  perturbate  the  target  shape.  Such  growths  were  observed  on  the  Key  West  targets  and 
present  a  very  challenging  identification  problem. 

The  target  cuer  concept  is  currently  being  coded  in  Microsoft  Visual  C++  to  run  on  a  PC.  Supporting 
code  has  been  developed  to  convert  various  laser  line  scan  system  image  formats  into  a  binary  matrix 
that  can  readily  be  input  into  the  cuer  and  target  identifier  test  programs. 

RESULTS 

Survey  Phase: 

Although  the  survey  weather  was  excellent  and  the  sea  state  mild,  the  imaging  conditions  within  the 
water  were  rendered  poor  by  what  could  best  be  described  as  an  algal  “snowstorm”  in  which  large 
coagulated  particle  groups  (on  the  order  of  one -half  inch  in  diameter)  streamed  through  the  water 
column  supported  by  strong  currents.  Water  measurements  have  yet  to  be  quantified  but  visibility  was 
estimated  at  one-two  meters.  The  FILLS  sensor  output  indicated  that  fluorescence  phenomena  were 


practically  negligible  in  the  natural  background  and  on  the  targets.  Only  low  levels  of  fluorescence 
were  visible  in  the  panel  image,  which  will  be  calibrated  by  CSS  in  controlled  water  at  a  later  date. 

Data  Analysis  Phase: 

Mine  imagery  obtained  revealed  a  high  level  of  bio-fouling  including  but  not  limited  to  sponge  growth 
and  a  fluffy  algal  coating.  Very  few  man-made  surfaces  were  evident  in  the  diver  inspections.  The 
degree  of  fouling  and  resulting  perturbations  to  object  shape  made  it  nearly  impossible  to  distinguish 
the  spherical  mine  from  the  box-shaped  anchor  in  the  top-down  FILLS  scenes.  The  prospects  for 
positive  object  identification  under  these  conditions  are  difficult  to  assess,  especially  without  a  larger 
sample  set  to  understand  the  feature  variability.  At  best,  the  identification  process  is  reduced  to 
detecting  “blobs”  within  a  defined  size  and  shape  range  but  with  varied  detailed  and  gross  surface 
structure  and  as  such  discriminating  between  a  fouled  mine  and  a  fouled  concrete  block  or  other  mine¬ 
sized  structure  is  not  possible.  Of  course,  if  it  were  known  that  only  highly  fouled  mines  and/or  their 
anchors  were  present  in  otherwise  benign  clutter,  a  special  case  strategy  could  be  devised  for  searching 
this  particular  environment,  but  pursuing  that  application  at  this  time  should  not  be  a  priority. 

IMPACT/APPLICATION 

There  is  no  indication  yet  that  fluorescence  cues  can  be  exploited  to  aid  automated  or  even  manual 
detection  or  identification  processes  as  applied  to  long  tenn  emplaced  heavily  bio-fouled  mines.  The 
test  conditions  may,  however,  have  been  inherently  non-conducive  to  propagating  fluorescence  signals 
and  may  not  be  representative  of  any  broader  class  of  waters.  Additionally,  the  bio-fouling  was  so 
extreme  that  its  impact  on  target  shape  and  signature  may  have  prevented  recognition  under  even  the 
best  of  imaging  conditions.  From  these  two  perspectives,  our  data  collection  was  “worst-case”  and 
does  not  support  general  conclusions  about  fluorescence  utility. 

As  an  immediate  consequence,  future  analysis  efforts  on  this  project  will  concentrate  primarily  on  data 
sets  with  less-fouled  targets  of  interest  from  deployments  in  other  environments  that  span  a  range  of 
water  conditions  from  clear  to  highly  turbid. 

TRANSITIONS 

Successful  and  robust  automated  target  cueing  and  identification  techniques  developed  under  this 
project  have  insertion  potential  into  Navy  programs  such  as  AQS/20X  and  RMS. 

RELATED  PROJECTS 

1  -  Mike  Strand  (CSS)  discusses  the  Key  West  survey  in  greater  detail  in  “Survey  of  a  WWII  Derelict 
Minefield  with  a  Fluorescence  Laser  Line  Scan  Sensor”.  This  report  also  discusses  the  real-time 
implementation  of  CSS  developed  image  enhancement  and  target  cueing  processes. 

2  -  Yensch  and  Phinney  (Bigelow  Labs)  in  “The  Fouling  of  Mine  Casing  Surfaces  by  Fluorescent 
Organisms”  are  investigating  shorter  term  and  seasonal  bio-fouling,  and  accompanying  fluorescence 
signatures,  on  naval  targets  in  Maine  and  Florida  littoral  regions. 
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